HOMOGENEITY RANK OF REAL REPRESENTATIONS 
OF COMPACT LIE GROUPS 



CLAUDIO GORODSKI AND FABIO PODESTA 

Abstract. The main result of this paper is the classification of the real irreducible repre- 
sentations of compact Lie groups with vanishing homogeneity rank. 



1. Introduction 

Let a compact Lie group G act smoothly on a smooth manifold M. The codimension of 
the principal orbits in M is called the cohomogeneity cohom(G, M) of the action. Piittmann, 
starting from an inequality for the dimension of the fixed point set of a maximal torus in G 
due to Bredon ([Bre72j, p. 194), introduced in |Pue02j the homogeneity rank of (G,M) as 
the integer 



homrk(G, M) = rk G - rk G princ - cohom(G, M) 

= rk G — rk Gprinc + (dim G — dim G w - mc ) — dim M, 

where G pr i nc is a principal isotropy subgroup of the action and, for a compact Lie group K, 
rk K denotes its rank, namely the dimension of a maximal torus. We will see in the next 
section that orbit-equivalent actions have the same homogeneity rank. 

This invariant, although not with this name, had already been considered by Huckleberry 
and Wurzbacher who proved that a Hamiltonian action of a compact Lie group on a symplec- 
tic manifold has vanishing homogeneity rank if and only if the principal orbits are coisotropic 
with respect to the invariant symplectic form (see [HW90| . Theorem 3, p. 267 for this result 
and other characterizations of this property). If p : G — > U(V) is a complex representation 
where V is a complex vector space endowed with an invariant symplectic structure, then the 
G-action is automatically Hamiltonian, and it has vanishing homogeneity rank if and only 
if every principal orbit is coisotropic; this condition can be proved to be equivalent to the 
fact that a Borel subgroup of the complexified group G c has an open orbit in V, and also to 
the fact that the naturally induced representation of G on the ring of regular functions C[V] 
splits into the sum of mutually inequivalent irreducible representations (see e.g. [Kra85j, 
p. 199). Complex representations with these equivalent properties are called coisotropic or 
multiplicity-free; Kac |Kac80| classified the irreducible multiplicity- free representations and, 
later, Benson and Ratcliff [BR96 and, independently, Leahy |Lea98j classified the reducible 
ones. 

In this paper we consider the case of an irreducible representation p : G — > 0(V) of a 
compact Lie group G on a real vector space V with vanishing homogeneity rank. Since 
representations admitting an invariant complex structure have null homogeneity rank if and 
only if they are multiplicity-free, we will deal only with irreducible representations of real 
type, also called absolutely irreducible, namely those which admit no invariant complex 
structure. Our main result is the following theorem. 
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Theorem 1. An absolutely irreducible representation p of a compact connected Lie group 
G has vanishing homogeneity rank if and only if it is either orbit- equivalent to the isotropy 
representation of a non-Hermitian symmetric space of inner type or it is one of the following 
representations: 



G 


P 


d 


c 


Sp(l) x Sp(n), n > 2 


(standard) 3 ®h (standard) 


8n 


3 


SO(4) x Spin(7) 


(standard) ®r (spin) 


32 


5 


Sp(l) x Spin(ll) 


(standard) £g>H (spin) 


64 


6 



where d denotes the dimension of the representation space and c denotes its cohomogeneity. 



2. Preliminaries 

Let (G, V) be an absolutely irreducible representation of a compact Lie group G on a real 
vector space V. It is shown in Corollary 1.2 in Pue02j that the homogeneity rank of a linear 
representation is non positive. In this regard, the representations with vanishing homogene- 
ity rank are precisely those with maximal homogeneity rank. The following monotonicity 
property that is stated on p. 375 in |Pue02| and is valid for smooth actions on smooth man- 



ifolds will be the basis of the method of our classification. Since there is no proof in |Pu e02j. 
we include one for the sake of completeness. 

Proposition 2. Let (G, M) be a smooth action. If G' be a closed subgroup of G, then 
homrk(G',M) < homrk(G,M). 

Proof. We first prove the statement in the case in which M is G-homogeneous, i. e. we 
prove that given a homogeneous space M = G/H, where G is a compact Lie group and H 
is a closed subgroup, for every closed subgroup G' of G we have 

homrk(C7', G/H) < rk G - rk H. 

We prove this by induction on the dimension of the manifold, the initial case dimM = 1 
being clear. Fix the point o = [H] e G/H, a maximal torus Tjj of H, and a maximal 
torus T of G containing Tjj. Since conjugation of G' by elements of G does not affect the 
homogeneity rank, we can assume that a maximal torus T' of G' sits inside T. Then we have 

rk G' — rk G' a < dimT' — dim(T' D G' ) = dimT' • o < dimT • o, 

where G' a denotes the isotropy subgroup of G' at o. Therefore 

rk G' - rk G' a <ikG-rkH. 

We now consider the slice representation of G' a on the normal space W to the orbit G' ■ o; 
we can assume that the dimension k of W is at least 2, since otherwise G' Q contains a 
principal isotropy subgroup of (G,G/H) as a subgroup of finite index and the claim follows 
immediately. Denote by S the unit sphere in W with respect to a G^-invariant inner product 
in W and apply the induction hypothesis. Since G' is a closed subgroup of SO(fc), we have 

homrk(G' ,S) = rk G' Q - rk Gp rinc - cohom(G', M) + 1 
< homrk(SO(A;),S) 
l + (-l) k 



where G' pTinc denotes a principal isotropy subgroup of G' on M. It then follows that 

homrk(G", M) < rk G' - rk G' Q + ~ l + ^ ~^ < rk G - rk H, 

and our claim is proved. 

In the general case, we fix a G-regular point p € M and observe that a point g G G ■ p is 
principal for the G'-action on G ■ p if and only if it is principal for the G'-action in M; this 
means that cohom(G", M) = cohom(G, M) + cohom(G", G ■ p). We know from the previous 
case that 

homrk(G", G ■ p) < homrk(G, G ■ p), 

and now our claim follows by subtracting cohom(G, M) from both members of the above 
inequality. □ 

Corollary 3. Let (G, V) be a representation of a compact Lie group G on a real vector space 
V . If (G, V) is not of vanishing homogeneity rank, then the action of a closed subgroup of 
G on V is never of vanishing homogeneity rank. 

The preceding corollary indicates a strategy to classify representations with vanishing 
homogeneity rank. First we observe that the standard representation of SO(n) on R n is 
of vanishing homogeneity rank if and only if n is even. Then we need to decide which of 
the maximal subgroups of SO(ra), where n is even, act absolutely irreducibly on R n with 
vanishing homogeneity rank. For each example that we encounter, we examine which of 
its maximal subgroups still act absolutely irreducibly on R n with vanishing homogeneity 
rank, and so on. The process will eventually yield all the closed subgroups of SO(n) that 
act absolutely irreducibly on R n with vanishing homogeneity rank. The effectiveness of this 
strategy is elucidated by the following well known result of Dynkin Dyn52a . 

Theorem 4 (Dynkin). (1) Let G be a maximal connected subgroup o/SO(n). Then G 
is conjugate in 0(n) to one of the following: 

(a) SO(k) x SO(n - k), where 1 < k < n - 1; 

(b) p(SO(p) x SO(g)) ; where pq = n and 3 < p < q, and p is the real tensor product 
of the vector representations; 

(c) U(fc), where 2k = n; 

(d) p(Sp(p) x Sp(g)) ; where Apq = n^4, and p is the quaternionic tensor product 
of the vector representations; 

(e) p(Gi), where G\ is simple and p is a real form of a complex irreducible repre- 
sentation of degree n of real type. 

(2) Let G be a maximal connected subgroup o/SU(n). Then G is conjugate to one of the 
following: 

(a) SO(n); 

(b) Sp(A;) ; where 2k = n; 

(c) S(U(Jfe) x U(n - k)), where 1 < jfe < n - 1; 

(d) p(SXJ(p) x SU(g)), where pq = n and p > 3 and q > 2, and p is the complex 
tensor product of the vector representations; 

(e) p[G\), where G\ is simple and p is a complex irreducible representation of degree 
n of complex type. 

(3) Let G be a maximal connected subgroup o/Sp(n). Then G is conjugate to one of the 
following: 
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(a) U(n); 

(b) Sp(/c) x Sp(n — k), where 1 < k < n — 1; 

(c) p(SO(p) x Sp(g)) ; where pq = n and p > 3 and g > 1, and p «s t/ie rea/ tensor 
product of the vector representations; 

(d) p(Gi), where G\ is simple and p is a complex irreducible representation of degree 
2n of quaternionic type. 

Recall that a symmetric space of compact type X = L/G is said to be of inner type if 
rk L = rk G; otherwise, X is said to be of outer type (compare Theorem 8.6.7 on p. 255 
in |Wol84j ) . Moreover, the isotropy representation of X is absolutely irreducible if and only 
if X is non-Hermitian. The following lemma implies that the isotropy representations of 
symmetric spaces of semisimple type that have vanishing homogeneity rank are precisely 
those coming from non-Hermitian symmetric spaces of inner type. 

Lemma 5. Let (G, V) be the isotropy representation of a symmetric space of compact type 
X = L/G. Then homrk(G, V) = if and only if rk G = rk L. 

Proof. Let I = g + V be the Cartan decomposition of X with respect to the involution, 
where [ and g respectively denote the Lie algebras of L and G. Let a C V be a max- 
imal Abelian subspace. By the structural theory of symmetric spaces, it is known that 
the dimension of a is equal to the cohomogeneity of (G,V), and that the centralizer m 
of a in g is the Lie algebra of a principal isotropy subgroup of (G,V). It follows that 
homrk(G, V) = rk q — rk m — dim o. Let t be a Cartan subalgebra of m. Then it is easily 
seen that t + a is a Cartan subalgebra of [. Now rk m = dim t, rk I = dim t + dim a, and 
hence homrk(G, V) = rk q — rk I = rk G — rk L which proves our thesis. □ 

We will also use Theorem 1.3 of |Pue02j which, for convenience of the reader, we restate 
here. 

Theorem 6 (Piittmann). Let (G,M) be an isometric action of the compact Lie group G on 
a Riemannian manifold M . Then, for any x G M , we have 

dimz/^G ■ x) Gx < cohom(G, M) - (rk G x - rk Gp rinc ), 

where v x [G ■ x) denotes the normal space to the orbit G ■ x at x, v x {G ■ x) Gx denotes the fixed 
point subspace of G x in v x {G ■ x), and G pr i nc is a principal isotropy subgroup of (G, M). 

The following proposition implies that orbit-equivalent actions have the same homogemeity 
rank. 

Proposition 7. Let (G,M) be a smooth action. If G' is a closed subgroup of G, and G and 
G' have the same orbits in M, then homrk(G', M) = homrk(G, M). 

Proof. It is clearly enough to prove that if G' and G act transitively on the same manifold 
M, then homrk(G",M) = homrk(G,M). If we represent M = G/H = G'/H' for suitable 
closed subgroups H C G and H' C G', then we claim that 

rk G - rk H = rk G' - rk H' . 

This follows from the fact that, given a homogeneous space M = G/H with G compact, the 
number Xn{M) := rk H — rk G is a homotopy invariant of M (see [Oni94], p. 207) □ 

Finally, we state the following direct consequences of the definition of homogeneity rank, 
which we shall repeatedly use in our arguments. 
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Remark 8. Let (G, M) be a smooth action of a compact Lie group G on a smooth manifold 
M. Then: 

(a) If homrk(G, M) = 0, then dimM < dimG + rk G. 

(b) If G' is a connected closed subgroup of G having the same homogeneity rank, then 
rk G' > rk G — rk Gprmc- Moreover if Gprmc is finite, then G = G' (indeed, G and 
G' are orbit-equivalent both with finite principal isotropy, hence G and G' have the 
same Lie algebra). 

3. The classification 

In this section, we apply the strategy discussed in the previous section to classify absolutely 
irreducible representations with vanishing homogeneity rank. It is enough to consider orthog- 
onal representations of even degree 2n. According to Theorem the maximal connected 
subgroups of SO(2n) acting absolutely irreducibly on V = R 2n are: p(SO(p) x SO(g)), 
where pq = 2n and 3 < p < q, and p is the real tensor product of the vector representations; 
p(Sp(p) x Sp(g)), where Apq = 2n ^ 4, and p is the quaternionic tensor product of the vector 
representations; and p(G\), where G\ is simple and p is a real form of a complex irreducible 
representation of degree 2n of real type. 

3.1. The case of p(SO(p) x SO(g)) and its maximal subgroups. Herepg = 2n and 3 < 
p < q. We have that p is the isotropy representation of the symmetric space SO(p + g)/SO(p) x 
SO(g), and rk SO(p + q) = rk SO(p) +rk SO(g) because not both of p, q are odd. It follows 
from Corollary El that this is an example. 

Next we must investigate maximal connected subgroups G of p(SO(p) x SO(g)). We shall 
consider separately three cases which cover all the possibilities. 

3.1.1. G = p{G\), where G\ = K xSO(g) , and K C SO(p) is a maximal connected subgroup. 
Set Gi = SO(p) x SO(g), G = p{Gi). There is a Gi-regular point x € V whose connected 
principal isotropy subgroup is given by Gi x = SO(q — p) C SO(g). The isotropy subgroup 
of Gi at x is the intersection Gi x fl G\, and its connected component is SO(g — p). If G has 
vanishing homogeneity rank on V, then Theorem El applied to x gives 

dim u x (Gx) Gx < cohom(G, V) — (rk G x — rk G pr inc) 
= rkG-rkO, 

= rkiT + rkSO(g)-rkSO(g-p) 
= rkiT + rkSO(p), 

where we have used that not both of p, q are odd. Note that dimz/^Gx) = p and 
dimz/j,(Gx) = p + dimSO(p) — dim.fr. It is clear that 

v x (Gx) G * D u x (Gx), 

since v x {Gx) D ^(Gi), G x C G x and x is G- regular. It follows that 

dim v x [Gx) Gx > p. 

Combining with the above we get that 

rk K + rk SO(p) > p > 2rk SO(p), 

and therefore 

ikK = rk SO(p). 
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By the classification of maximal subgroups of maximal rank of SO(p), (see, for example, 
section 8.10 in jWo!84j ). since K is irreducible and of real type on R p , we must have K = 
SO(p). 

3.1.2. G = p(Gi), where G\ = SO(p) x K , and K C SO(g) is a maximal connected subgroup. 
According to Theorem HJ we need to consider three cases. 

(a) K = p(K\), where K\ is simple and p is an absolutely irreducible representation of 
degree q. Of course we need only to consider representations p such that p{K\) is a proper 
subgroup of SO (5). 

We may assume p < q. Remark Eta) gives that p 2 < pq < si2zll _|_ |jj + r, where 
r = dimi^i + rk K\, and [x] denotes the greatest integer contained not exceeding x. This 
implies that 

(1) p 2 < 2r, 
and that 

(2) p 2 - 2qp + 2r > 0. 
Equation (J2J) and p > 3 then imply that 

r 3 

(3) q <- 3+ - r 

Let s be the minimal degree of an absolutely irreducible representation of K and such that 
its image is not the full SO(s). Then 

fA\ r 3 

(4) s > - + - 
v ; 3 2 

is a sufficient condition for (G, V) not to be of vanishing homogeneity rank. We next run 
through the possibilities for K\. 

• K\ = SU(m), where m > 2. Here r = m 2 + m — 2. If m = 2, then s — 5, and Pjl 
holds. If m > 3, then s = m 2 — 1 (realized by the adjoint representation), and (jlj) 
holds. 

• Z£i = Sp(m), where m > 2. Here r = 2m 2 + 2m. If m = 2, then s = 10 (realized by 
the adjoint representation) and Q holds. If m > 3 then s = 2m 2 — m — 1 (realized 
by the second fundamental representation) and holds. 

• Ki = Spin(m), where m > 7. Here r = m (™~ 1 ) _|_ j^aj _ All irreducible representations 
of real type violate (j3J, except possibly the (half-)spin representations of B 4 fc_i, B 4 fc, 
D 4 fc. These have respectively q = 2 4fc_1 , 2 4fc , 2 4/c_1 . Condition © is respectively 

3 • 2 4fc < QAk 2 - 16k + 9, 

3 • 2 4fc+1 < 6Ak 2 + 16k + 9, 

3 • 2 4fe < 64k 2 + 9. 

The only cases that survive are B 3 and D 4 . In the case of B 3 we have q = 8 
and r = 24. Then (JTJ) implies that p — 3, 4, 5, 6. Next we use (J2J) to get rid of 
p — 5, 6. We end up with p = 3 and p = 4, and this gives the admissible cases 
(SO(3) x Spin(7),R 3 ®R 8 ) and (SO(4) x Spin(7),R 4 ®R 8 ), but note that the first 
one of these is orbit-equivalent to (SO (3) x SO (8), R 3 ® R 8 ). In the case of D 4 , we 
have that yu(Spin(8)) = SO(8), and we rule this out. 
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K\ is an exceptional group. Here (@J holds in each case, so there are no examples, 
see the table below. 





r 


s 


G 2 


16 


7 


F 4 


56 


26 


E 6 


84 


78 


E 7 


140 


133 


E 8 


256 


248 



(b) K = p(SO(k) x SO(/)), where 3 < k < I and q = kl, and /i is the real tensor product 



of the vector representations. Here r 



k 2 +i 2 



k 

2 + 



-. Note that -1< 6 < 0. 
2 ~ ~ 



Then © is k 2 + l 2 - Qkl + 26 + 9 > 0. Set m = l-k > 0. Then m 2 - 4km - Ak 2 + 9 + 26 > 0. 
This implies that 



(5) 



2k + V8k 2 -9-26 < 



m. 



If pk < I, since the action of SO(p) x ^(SO(k) x SO(/)) C SO(p) x SO(g) on R p ® R 9 is 
the same thing as the action of //(SO(» x SO(fc)) x SO(/) C SO(pA;) x SO(Z) on R pfe ®R\ 
where fi' is the real tensor product of the vector representations, this case has already been 
considered in section 13.1.11 So now we assume that 



(6) 



pk > I. 



Note that q > \[2x. This implies via (j2j) that 3 < p < q — \J q 2 — 2r. Combining this with © 
we have I < kq — k\J q 2 — 2r and then l 2 (l — 2/c 2 ) + 2k 2 r > 0. Substituting the value of r we 

get k < I < kj^f. We deduce that 6 = and 



(7) 



< m < k 



k 



Now (jSJ) and (J7|) combined imply that 3k + y/8k 2 — 9 < -yp=p which is impossible for > 3. 
(c) K = fi(Sp(k) x Sp(7)), where q = Akl ^ 4, and /i is the quaternionic tensor product of 
the vector representations. We postpone this case to section 13.41 



3.1.3. G = {(x,a(x)) : x G SO(p)}, where p = q and a is an automorphism ofSO(p). Here 
Remark |Hfa) immediately implies that (G, V) cannot have vanishing homogeneity rank. 



3.2. The case of p(Sp(p) x Sp(g)) and its maximal subgroups. Here 4pq = 2n ^ 
4 and p < q. We have that p is the isotropy representation of the symmetric space 
Sp(p + q)/Sp(p) x Sp(g), and rk Sp(p + q) = rk Sp(p) + rk Sp(g). It follows from Corol- 
lary El that this is an example. 

Next we must investigate maximal connected subgroups G of p(Sp(p) x Sp(g)). We shall 
consider three cases separately which cover all the possibilities. 
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3.2.1. G = p(G\), where G\ = K x Sp(g), and K C Sp(p) is a maximal connected subgroup. 
Set G\ = Sp(p) x Sp(g). There is a Gi-regular point x G V whose principal isotropy 
subgroup is given by G\ x = Sp(l) p x Sp(g — p). Let O be the orbit of G\ through x. A 
point y 6 O is Gi-regular in O if and only if it is Gi-regular in V; moreover the isotropy 
subgroup Gi y is given by the intersection of Gi with a suitable conjugate of G lx in G\. This 
means that a principal isotropy subgroup of G\ contains a subgroup isomorphic to Sp(g — p). 
If (G, V) has vanishing homogeneity rank, it then follows that 

dimV^ = Apq < dimK + rk K + 2q 2 + 2q - (2(q - pf + 2(q-p)), 

hence 

(8) 2p 2 - 2p < dim K + rk K. 

According to Theorem 01 there are two cases to be considered. But if K is of the form 
/i(SO(&;) x Sp(/)), where \i is the real tensor product of the vector representations, we refer 
to section l3~^l So we can assume that K is of the form fi(Ki), where K\ is simple and \i 
is a complex irreducible representation of degree 2p of quaternionic type. Let s be half the 
minimal degree of a complex irreducible representation of quaternionic type of K\ and such 
that its image is not the full Sp(s). The list of the values of s for each compact simple 
group K\ is given by the following table (groups not appearing in the table do not admit 
quaternionic representations): 





s 


SU(4a + 2), a > 1 


1 /4a+2\ 

2 \2a+l) 


Spin(8a + 3), a > 1 


2 4a 


Spin(8a + 4), a > 1 


2 4a 


Spin(8a + 5), a > 1 


2 4a+l 


Sp(a), a > 3 


f(2a 2 -3a-2) 


Sp(2) 


8 


Sp(l) 


2 


E 7 


28 



It is now easy to see that (jH)) implies Ki = Sp(l) and p = 2, and then the only admissible 
case is G\ = Sp(l) x Sp(g), where Sp(l) C Sp(2) via the irreducible representation of 
degree 4. 



3.2.2. G = p{Gi), where G\ = Sp(p) x K , and K C Sp(g) is a maximal connected subgroup. 
According to Theorem we need to consider two cases. 

(a) K = fi(Ki), where K\ is simple and fi is a complex irreducible representation of degree 2q 
of quaternionic type. We may assume p < q. Remark|Hfa) gives that 4p 2 < 4pq < 2p 2 + 2p+r, 
where r = dimi^i + rk K\ (note that r > 4). This implies that 



p < ^ and 2p 2 + 2p(l - 2q) + r > 0. 
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V 



and 



(10) 



if q > 



l+v / l+2f 
2 



, then p < q — \ — \ 



iV(2g-l)2-2r. 



Running through the compact simple groups K\ that admit quaternionic representations 
(see table in section 13.2. 1J) and using (0) and (fTUj). we get the following admissible cases: 



K x = Sp(l), p=l,q = 2]K 1 = Sp(3), p = 1, g = 7; JTi = Spin(ll), p = 1, g = 16; 
K x = Spin(12), p = 1, q = 16; K x = SU(6), p = 1, g = 10; i^i = E 7 , p = 1, q = 28. All 



cases but that of K\ = Spin(ll) come from isotropy representations of symmetric spaces, 
(b) K = fi(SO(k) x Sp(/)), where q = kl, and /i is the real tensor product of the vector 
representations. We postpone this case to section IX-El 

3.2.3. G = {(x,a(x)) : x G Sp(p)}, where p = q and o is an automorphism o/Sp(p). Here 
Remark E^a) immediately implies that (G, V) can have vanishing homogeneity rank only if 
p = 1, so this case is out. 

3.3. The case of p(G\). Here G\ is a compact simple Lie group and p is an absolutely 
irreducible representation of G\ of degree 2n. Remark Ufa) says that 2n < dimG*! +rk G±. 
In particular, this implies that 2dimGi > 2n — 2, so we can use Lemma 2.6 in |Kol02j to 
deduce that (G, V) is orbit equivalent to the isotropy representation of a symmetric space. 

3.4. The case of p(SO(m) x Sp(p) x Sp(g)), where p is the real and quaternionic 
tensor products of the vector representations. Here 2n = 4mpq, m > 3 and p < q. 
By direct computation or using Theorem 1.1 in |HH70j . we see that: 

(i) if m > 4pg + 2, then the connected principal isotropy is given by SO(m — 4pg); 

(ii) if g > mp + 1, then the connected principal isotropy is given by Sp(g — mp); 
(hi) in all other cases the connected principal isotropy is trivial. 

In case (i) the condition of vanishing homogeneity rank reads 



If m = 21, then we have p 2 (l — 4/ 2 ) +p(l + 21) + 1 2 = 0, which implies that 1 + 2/ divides I 2 , 
impossible. If m = 21 + 1, then we have (4p 2 — 1)/ = 2p, which is impossible. In case (iii), 
we have the equation 



4p 2 g 2 = p 2 + q 2 + p + q < 2p 2 + 2q 2 , 
and this implies p = q = 1 . In case (ii) we have 

4p 2 (l - m 2 ) + 4p(l + m) + m 2 - m + 2 — 



0. 



8mpq 



m 



2 - m + 2 — + 4p 2 + 4p + 4g 2 + 4g. 




4pgZ + p 2 + q 2 + p + q = 0, 



— <l< 2pg, p < g, / > 2, 




I 2 - (4pg - 1)/ + p 2 + g 2 - 2pg + p + g = 0, 
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subject to the constraints 

q 1 

< I < 2pq, p<q, > 1. 

Consider first equation (fTTj) . It can be solved in / to yield / = 2pq ± v^A, where A = 
Ap 2 q 2 — p 2 — q 2 — p — q. If Z = 2pq + v^A, using the fact that Z < 2pq we have A = and 
then I = 2pq < p + q, which gives p = q = 1, and then I — 2, m — A. If Z = 2pg — v^A, then 
^ < / implies that 

q 2 (Ap 2 - 1) - 4p 2 g - Ap 2 (p 2 + p) < 0, 

and therefore 

2p 2 + 2 Pv /p{Ap 3 + Ap 2 - 1) 
p<g< ^— <p + 2. 

It then follows that we only need to consider the possibilities q = p and q = p + 1. If g = p, 
then we have 

2 < / = 2p 2 - V V - 2p 2 - 2p < 2, 
so that I = 2, p = q = 1. li q = p + 1, then 



2 < I = 2p 2 + 2p - v^p 4 + 8p 3 + 2p 2 - Ap - 2 < 2, 

which is impossible. 

Next we consider equation (j!2|) . Here it is useful to note that Remark |Hfb) applied to 
G = SO(m) x Sp(p) x Sp(q r ) viewed as a subgroup of SO(m) x SO(Apq) gives the extra 
condition 

(13) [-\<P + q. 

Equation (fT2j) can be solved in Z to yield Z = ^pbh^/AL^ where Ai = \Qp 2 q 2 — Ap 2 — Ap — 
Aq 2 - Aq + 1. If / = tetidVA^ then ^ implies that 

2pq < I < p + q < 2q, 

which gives p = q = 1, Ai = 1, Z = 2 and m = 5. If / = 4p9 ~ 1 ~^ 5 ^ , then the inequality 
£ — ^ < Z implies that 

q 2 (Ap 2 - 1) - 4p 2 g - 4p 4 - Ap 3 + p 2 <0, 

which in turn implies 



2p 2 + pJVop^ + 16p 3 - Ap 2 - Ap + 1 
p<g< ^— <p + 2. 

Therefore we need only to consider the cases q = p and q — p + 1 . If g = p, then 

^ _ 4p 2 - 1 - v/l6p 4 - 8p 2 - 8p + 1 
~ 2 < 

This gives I = 1, m = 3, p = q = 1. If g = p + 1, then 

Z = 2p 2 + 2p - - - - ^/I6p 4 + 32p 3 + 8p 2 - 16p - 7 < 1, 
which is impossible. 
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3.5. The examples and their subgroups. In this section we show that all candidates 
G C SO(2n) found in the previous sections are actually examples of groups acting absolutely 
irreducibly on V = R 2n with vanishing homogeneity rank, and these groups do not admit 
subgroups with the same property. This will complete the proof of Theorem ^ 

We first examine the representations that are not orbit-equivalent to isotropy representa- 
tions of non-Hermitian symmetric spaces of inner type. We have three candidates: 

(1) G = Sp(l) x Sp(g) (q > 2) acting on V = S 3 {C 2 ) © H C 2q R 8g ; 

(2) G = SO(4) x Spin(7) acting on V = R 4 ® R 8 = R 32 , where Spin(7) acts on R 8 via 
the spin representation; 

(3) G = Sp(l) x Spin(ll) acting on V = C 2 © H C 32 S R 64 , where Spin(ll) acts on 
C 32 via the spin representation. 



We now show that in each case the representation has vanishing homogeneity rank. Indeed, 
in case 1 we have that a connected principal isotropy is given by Sp(g — 2) (see GTOOJ, 



Proposition 7.12), therefore the cohomogeneity is three and the homogeneity rank vanishes. 
In connected isotropy subgroup is trivial. This can be seen by selecting a pure 

tensor v ® w with v E R 4 and w E R 8 and computing the connected isotropy, which is 
SO(3) x G2; then the slice representation is given by R©R 3 ©R 7 ; starting again with this new 
representation, we eventually come up with a trivial isotropy. Therefore the cohomogeneity 
is five and the homogeneity rank vanishes. In case 3, we also have trivial connected principal 
isotropy and vanishing homogeneity rank. Indeed, if v E C 32 is a highest weight vector for 
the spin representation of Spin(ll), then the subgroup H C Spin(ll) defined by H = {g E 
Spin(ll) : g ■ v E C* • v} is given by U(5). Now if p : Sp(l) x Spin(ll) — > Spin(ll) is the 
projection, then 



p((Sp(l) x Spin(ll))„) = {g E Spin(ll) : g • v E Sp(l) -v}dH. 



Since H is maximal in Spin(ll), we get that (Sp(l) x Spin(ll))„ is given by T 1 • SU(5), 
where T 1 sits diagonally in the product of a suitable maximal torus in Sp(l) and the center 
of H. From this we see that the slice representation at v is given by R © C 5 © A 2 C 5 and the 
connected principal isotropy is trivial. The cohomogeneity is six and the homogeneity rank 
vanishes. 

We now examine subgroups of the previous examples. In case 1, a maximal subgroup 
of G leaving no complex structure on V invariant is of the form G' = Sp(l) x K, where 
K C Sp(g) is maximal. Since Sp(2) x K does not have vanishing homogeneity rank on 
V by the results of section 13.2.21 and G' C Sp(2) x K, we have that G' does not have 
vanishing homogeneity rank on V. In cases 2 and 3, G admits no proper subgroups acting 
with vanishing homogeneity rank because the connected principal isotropy is trivial and then 
we may apply Remark |BJb). 

We finally consider the representations (G, V) that are orbit-equivalent to isotropy repre- 
sentations of non-Hermitian symmetric spaces of inner type, and we classify the subgroups 
G' C G which still act absolutely irreducibly on V with vanishing homogeneity rank. In the 
following table we list the representations p which need to be examined; we denote by c the 
cohomogeneity of p, by d the dimension of V, and by [[W]} a real form of the G-module W. 
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Q 
O 


1 98 
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n 
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SU(8) 


[[A 4 C 8 ]] 


7 


70 





8 


SO(3) x Spin(7) 


R 3 ®R 8 


3 


24 


3 


9 


Spin(7) 


spin 


1 


8 


14 


10 


Spin(9) 


spin 


1 


16 


21 



Cases 4 through 7 can be dealt with using Remark IBJb). We consider case 1. If G' is 
a maximal subgroup of G, we may assume that G' is of the form G' = Sp(l) • G", where 
G" is maximal in SU(6), since G' does not leave any complex structure invariant. Now 
Remark Ufa) implies that dimG" + rk G" > 36, and rk G" < 5 implies dimG" > 31, so 
that dim SU(6)/G" < 4. If G" is a proper subgroup of SU(6), then the left action of 
SU(6) on SU(6)/G // is almost effective because SU(6) is simple. Therefore dimSU(6) is 
less than the dimension of the isometry group of SU(6)/G", which is at most 10, but this is 
a contradiction. Hence G" = SU(6). 

In case 2, again we can assume that G 1 is of the form G' = Sp(l) ■ G", where G" is 
maximal in Spin(12). We have dimG" + rk G" > 60; since G' is supposed to act absolutely 
irreducibly on V, its rank is not maximal by a theorem of Dynkin (see Theorem 7.1, p. 158 
in |Dyn5 2b|), and therefore dimG" > 55 = dim Spin(ll). It is known that a subgroup 
of Spin(n) of dimension greater or equal to dimSpin(n — 1) is conjugate to the standard 
Spin(n - 1) C Spin(n) if n ^ 4, 8 (see e.g. |Kob72j . p. 49). So, G' = Sp(l) • Spin(ll), 
which is indeed an example with trivial connected principal isotropy by the discussion above. 

In case 3, using the same argument as in case 2, we see that G' = Sp(l) • G" , where 
dimG" > 102. An inspection of the list of all maximal subalgebras of E7 (see Table 12, 
p. 150 and Theorem 14.1, p. 231 in |Dyn52b| ) shows that there is no such proper subgroup. 

In case 8, a maximal subgroup G' acting absolutely irreducibly on V must be of the form 
G' = SO(3) x K, where K C Spin(7) is maximal; arguing as above, we see that dimi^ > 18, 
so that dim(Spin(7)/K) < 3 and this is impossible, because Spin(7) is simple. 

In case 9, let K C Spin(7) be a maximal subgroup acting absolutely irreducibly on R 8 . 
Since K cannot have maximal rank as above, and using Theorem we see that K must be 
simple of rank at most two and it must admit an irreducible representation of degree 7 and 
of real type. Moreover, by Remark Ufa), we have dim K > 6, hence rk K = 2, and a direct 
inspection of all such simple groups shows that none of them but G2 admits an irreducible 
representation of degree 7. But G2 does not admit an irreducible representation of degree 8. 

In case 10, we consider a maximal subgroup K of Spin(9) acting absolutely irreducibly 
on R 16 . This means that rk K < 3 and dimf^ > 13. Looking at the list of all maximal 
subgroups of Spin(9), we see that we can suppose K to be simple and to act irreducibly on 
R 9 , via the embedding K C Spin(9). Therefore K must be one of G 2 , SU(4), Spin(7) or 
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Sp(3), but we immediately see that none of these groups admits an irreducible representation 
of degree 9 and of real type. 

This finishes the proof of Theorem ^ 
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